Cross-lingual alignments of ELMo contextual embeddings

Building machine learning prediction models for a specific NLP task requires sufficient training data, which can be difficult to obtain less-resourced languages. Cross-lingual embeddings map word from language resource-rich so that model trained on data the also used in language. To produce cross-lingual mappings of recent contextual embeddings, anchor points between embedding spaces have words same context. We address this issue with novel method creating alignment datasets. Based that, we propose several mapping methods ELMo embeddings. The proposed linear use existing Vecmap and MUSE alignments Novel nonlinear ELMoGAN are based GANs do not assume isomorphic spaces. evaluate nine languages, using four downstream tasks: named entity recognition (NER), dependency parsing (DP), terminology alignment, sentiment analysis. perform very well NER tasks, lower loss compared direct some In DP analysis, variants more successful.